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ABSTRACT 

Higher education has seen an increase in the number of faculty conducting schol¬ 
arship of teaching and learning (SoTL). Unfortunately, the momentum of this 
movement is somewhat thwarted by debates over which methodology should 
take center stage. The discussion often pitches quantitative experimentation 
against qualitative approaches and mistakenly represents each method as char¬ 
acterizing the domains of social science and the humanities, respectively. Does 
one methodology reign supreme? In this article I argue that not only are the hu¬ 
manities and the social sciences misrepresented, but both methodologies have a 
lotto contribute to SoTL. The social sciences and humanities, in fact, share many 
methodologies although they are each sometimes characterized as separate 
magisteria. I enumerate specific reasons why stereotypical social science meth¬ 
odology and that of the humanities are both useful for SoTL. I draw special at¬ 
tention to the time and place for statistical analyses and advocate for a 'fox-like' 
strategy to SoTL, one that involves utilizing mixed-methods research designs, 
and the collection of both quantitative and qualitative evidence. 
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Should faculty from the humanities know how to use social science methodologies 
to conduct scholarship of teaching and learning (SoTL)? Although this question is of¬ 
ten heard at faculty development conferences and workshops, it mistakenly stereotypes 
each discipline assuming all humanists use a qualitative approach and all social scientists 
use the quantitative approaches. Some SoTL faculty from the humanities and the arts 
consequently feel pressured to know how to design experiments and conduct statistical 
analyses. Pressure aside, “We do not care about p values or statistical significance. If you 
do, you calculate it!” is what some faculty retort. I argue that not only are methodologies 
varied within both the humanities and the social sciences, both qualitative and quantita¬ 
tive approaches have a lot to contribute to SoTL. 

SoTL is getting popular. There is a brand new journal for it coming out almost every 
year. Teaching and Learning Inquiry in 2013, SoTL in Psychology in 2014, join a host of 
other journals publishing research on teaching and learning (e.g., Teaching of Psychology, 
International Journal on the Scholarship of Teaching and Learning). Pat Hutchings, Mary 
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Taylor Huber, and Anthony Ciccone of the Carnegie Foundation for the Advancement 
of Teaching recently showcased how SoTLhas an impact on and has been integrated into 
higher education ( Scholarship of Teaching and Learning Reconsidered, 2011). 

SoTL entails methodologically rigorous scholarly work conducted to enhance teach¬ 
ing and advance learning. It is commonly described as intentional, systematic reflections 
on teaching and learning resulting in peer-reviewed products made public (Gurung & 
Schwartz, 2012; Potter & Kustra, 2011). Boyer popularized the term “scholarship of teach¬ 
ing” in his Scholarship Reconsidered, although caring teachers have practiced the kind of 
work it refers to for many years. Even though the extent to which SoTL is counted as 
scholarship towards merit, tenure, and promotion varies across the United States, more 
departments at more universities are including SoTL in faculty reviews. But is there a best 
way to do it? Does one discipline hold the key to the kingdom? 

Some of the rhetoric in SoTL presentations and publications does seem to imply 
that quantitative methodologies and the scientific method are the key ways to do SoTL 
(Maurer, 2011). For example, results from a survey of editors of SoTL journals suggested 
manuscripts did not get published because, among other reasons, they featured ‘inade¬ 
quate data analysis’ (Jarvis & Creasey, 2009). Alluding to pressures felt by some SoTL 
researchers to use the scientific method, Grauerholz and Main (2013) discuss how the 
scientific method cannot be fully used. Poole (2013) takes this discussion a step further 
to unpack what is (and what should be) meant by ‘research’ in the first place. Whereas 
benchmarks for SoTL in psychology are understandably explicitly empirically based 
(Wilson-Doenges & Gurung, 2013) a universal set of good practices for SoTL need not 
be and are not at all (Felten, 2013). 

In many ways these expositions resemble the age-old battle between the sciences and 
the humanities in general. Stephen Jay Gould famously argued that the two were ‘separate 
magisteria’ each with their own domains, not overlapping in anyway. One, social science, 
answered the how questions, the other, humanities, answered the ‘why’ questions. More 
recently, White (2013) flies the flag for the humanities calling science a ‘delusion’ and 
for a return to Romanticism. Whereas pitching the humanities against the social sciences 
makes for great intellectual debate, it is a sideshow and a distraction risking the derail¬ 
ment of advancing SoTL. One can spend a lot of time comparing various definitions of 
SoTL (Potter & Kustra, 2011 for a review) or one can just do it—focus on formulating 
questions about teaching and learning, educating oneself on what has been done on the 
topic, and the best methodologies to answer the question in a discipline-agnostic fashion. 

Most importantly, pitching the social sciences against the humanities mistakenly 
stereotypes the practitioners of each discipline. This apparent divide probably arose from 
the fact that there are methods and paradigms that the social sciences and humanities do 
not share. For example, the use of textual analysis or close reading is more common in the 
humanities, and the tendency toward more positivist beliefs in “truth” in more common 
in the social sciences. This notwithstanding, although the majority of social scientists do 
use quantitative methods, run statistical analyses, and examine probability values, it is in¬ 
accurate to assume that all social scientists use quantitative methods. Similarly, although 
many humanists take a constructivist approach and utilize qualitative methodologies never 
needing a statistical program or a calculator to analyze their data or evidence of student 
learning, it is inaccurate to assume all humanists only use qualitative approaches. Increas- 
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ingly, social scientists are using qualitative methods and approaches such as grounded 
theory and humanists are using quantitative methods such as content analysis. 

The reality is that there are many ways to gather knowledge, and methods of inquiry 
vary across disciplines (Creswell, 2014; Rubin & Rubin, 2012). Disciplines such as En¬ 
glish and History primarily take the hermeneutical approach, in which interpretation is 
important and textual meaning is constructed through a blend of understanding and ex¬ 
planation (Donald, 2002). In the social sciences, the scientific method and quantitative 
methods dominate although other methodologies are also used. Scientific approaches 
favor empirical demonstrations, the use of objective methods, and the replication of 
findings. The reality is that both these approaches have their uses and it behooves SoTL 
researchers to have at least a sense of both of them. 

Being skilled at one approach is commendable but may be limiting in that you may 
only ask the questions that your approach can study. Gould nicely draws our attention 
to an ancient Greek proverb that provides an apt warning: Multa novit vulpes, verum echi¬ 
nus unum magnum-The fox devises many strategies: the hedgehog knows one great and 
effective strategy (Archilochus in Erasmus, 1500, in Gould, 2003). If when trapped the 
hedgehogs’ one strategy, rolling into a ball, does not work, it is all over. The fox with many 
alternatives at her disposal fares better. Likewise, whether social scientist or humanist, 
the more strategies one has to find meaning, describe and explore learning, and examine 
the evidence, the better. SoTL practitioners have enough challenges and threats ranging 
from fighting for recognition of the scholarship as ‘good enough’ to finding time to do it 
and making it count. Why add the age-old battle between science and the humanities? 
Let’s be like the fox instead. 

Perhaps the real question is whether all SoTL researchers should learn quantitative 
methodologies? To a large extent the answer depends on the question. Different ques¬ 
tions call for different methods to get the best answer. There are three main reasons why 
quantitative methodology is useful and it should be noted that the methods and analyses 
mentioned are by no means restricted to the sciences. Quantitative methods, exemplified 
by the scientific method, involve a procedure or plan of action whereby one first observes 
a phenomenon, forms a theory that potentially explains the phenomenon, generates test¬ 
able guesses or hypotheses about the phenomenon based on the theory, designs a research 
study and collects evidence to test the hypothesis, and finally evaluates the hypothesis. 
This approach is referred to as a quantitative approach. Quantitative approaches are of¬ 
ten pitted against qualitative approaches whereby one studies the phenomenon in depth, 
taking detailed notes on it and gathering as much information about it from the sample of 
interest, and then looking for themes in what is gathered (Creswell & Piano Clark, 2012). 

So what can the social sciences contribute? First, social science research nicely alerts 
us to the fact that the changes we may see in our students’ learning maybe due to a whole 
host of factors, one of which may indeed be what we do as instructors or what the stu¬ 
dents have done (perhaps because of our instruction), but may also be likely due to other 
naturally occurring factors. People change over time (i.e., maturation). Factors outside 
our awareness influence results (i.e., history). It is possible that changes we see in our 
student’s learning are due to these natural changes or factors external to our instructional 
interventions. Social science methodology alerts us to these two confounds, to many oth¬ 
ers, and most importantly, to ways to avoid them. 
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Second, quantitative social science methodology involves significance testing. It is 
not enough to just observe change in student learning, but it is important to ask if that 
change was statistically significant. Assessing change is one type of pedagogical endeavor 
that necessitates the quantitative method regardless of discipline. If you want to know if 
students improved after a change you made (e.g., a new assignment, an innovative presen¬ 
tation, group work, flipping your class), you should know if that change would have hap¬ 
pened by chance or if any other factors could account for it. This requires quantification 
of the evidence (e.g., themes brought up in a close reading, concepts used in essays, levels 
of meaning). When social scientists ask if the change is statistically significant, they really 
want to ensure that the change is due to what was done and not just due to chance. Stated 
in this way, it seems hard to not care about statistical significance. If you have worked 
hard to change your instruction and improve student learning, it is important to know 
whether that change would have happened by chance and without your intervention in 
the first place. Before one spends more time and energy on changing instruction or even 
trying to get others to also change instruction based on the changes you have seen, you 
should be sure your changes are not random and would not have occurred naturally (e.g., 
a practice effect). Statistical testing does this for you. 

Statistical significance need not be the ultimate and only criterion for quality SoTL 
but it is certainly something to be considered for appropriate and relevant research de¬ 
signs and questions. Furthermore, statistical significance should not be confused with 
or taken to be synonymous with ‘significant’ as used in everyday life (i.e., to mean im¬ 
portant). Misinterpreting this term can easily make disciplines that do not use statistical 
testing feel unwelcome. There are many ways to conduct scholarship, make meaning, and 
demonstrate knowledge especially in the humanities and fine arts, which do not require 
statistical testing (Chick, 2013). 

Finally, having even a basic working knowledge of social science methodology helps 
us better digest the published literature in SoTL that may use this methodology. An expert 
SoTL practitioner should read the literature from a diverse array of disciplines. Being fa¬ 
miliar with the approaches of the different disciplines will both ensure critical thinking 
about the results of SoTL from the different disciplines but may also provide one with 
novel ways to approach one’s own SoTL. There are times when qualitative approaches 
provide key insights, and other times when quantitative approaches may be more suit¬ 
able to the pedagogical question raised. 

Starting with disciplinary methodologies and approaches you are comfortable with 
(and know how to use) makes great sense, but why stop there? If your focus is learning 
in general, it makes better sense to develop your question and then pick the best meth¬ 
odology for the question regardless of what discipline it comes from. This may involve 
collaboration and reading up on how other disciplines conduct SoTL (Chick, Haynie, & 
Gurung, 2012; Gurung, Chick, & Haynie, 2009). The question: Are my students learn¬ 
ing my disciplinary content is easily and better subsumed in the more general question: 
Are my students learning? 

To best unpack the complex enigma that is learning, we need to use the best tools pos¬ 
sible. This involves stepping outside our disciplinary comfort zones, crossing magesteria, 
to inform our SoTL. To foster a deeper understanding of the interconnections underlying 
learning, economists should know where historical data come from, sociologists should 
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know how to think like economists, political scientists would benefit from understanding 
how models are tested in psychology, historians should learn how political processes are 
studied, psychologists should understand sociological theories, and so forth (Gelman & 
Cortina, 2009). Having a sense of the different models and different ways of thinking in 
different disciplines informs one’s own scholarship. The question is not: “What discipline 
is doing the best SoTL?” or “Do I have to learn how to use methods I am not familiar 
with?” The question should be: What is the best way to answer my pedagogical research 
question? Not surprisingly, social scientists’ questions are more amenable to quantita¬ 
tive measures and humanist’s questions are more amenable to qualitative approaches. 
Humanists may not care about the questions that require the scientific method just as 
social scientists may not care about quests exclusively for meaning. 

The humanities have many sophisticated ways of answering questions. There is cer¬ 
tainly no need to shoehorn quantitative methodologies in where they are not apt. Often 
purely quantitative measures (e.g., responses to a priori created questions completed using 
a Likert-type scale) miss out on capturing the richness of experiences, thought processes, 
and conceptual paths followed by individuals. For example, few, if any quantitative mea¬ 
sures can get at what students see in a text, in the same way that use of close reading can 
(Brummett, 2010). Qualitative evidence makes learning processes and pitfalls transpar¬ 
ent in ways quantitative evidence cannot. 

Does a qualitative researcher need to know about quantitative research? My answer 
is that it can only help in general, but especially if examining changes in learning. By the 
same token, a quantitative scientist should know about qualitative methodologies as well. 
In many ways, this is the ‘greatest enterprise of the mind,... the attempted linkage of the 
sciences and the humanities’ (Wilson, 1998, p. 8). When you want to describe, explore, and 
examine something as complex as learning, nothing short of mixed-methods research— 
seamlessly blending quantitative and quantitative approaches and methodologies—will 
do the trick. Let’s be like the fox. 
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